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Introduction 


For  this  project,  we  have  primarily  addressed  the  problem  of  wireless  coverage  using  a  team 
of  mobile  robots  with  ad-hoc  communications  through  the  development  of  distributed 
probabilistic  coordination  models  and  algorithms.  For  coverage,  a  team  of  mobile  robots, 
each  with  limited  sensing  and  communications  ranges,  must  provide  a  set  of  communication 
chains  that  cover  as  large  a  space  as  possible.  Our  goal  was  to  develop  robot  coordination 
algorithms  that  perform  communications  chaining  accurately  and  robustly  in  the  face  of 
various  uncertainty  factors,  such  as  noise  in  sensing  and  wireless  signal  strength,  with 
secondary  consideration  of  constrained  resources,  such  as  power,  computation,  and 
communications  bandwidth.  We  proposed  and  developed  robot  software  to  reach  this  goal 
based  on  the  Markov  Random  Fields  (MRFs)  as  a  probabilistic  model  for  distributed 
inference.  The  Multi-robot  Markov  Random  Field  model  (described  in  [1]  and  [2])  allowed  us 
to  unify  existing  algorithms  for  multi-robot  coordination,  such  as  the  commonly  used  potential 
field,  in  a  generative  model  and  propose  new  coordination  algorithms  using  belief 
propagation.  Based  on  this  model,  we  produced  a  plug-and-play  software  framework  for 
distributed  multi-robot  coordination  on  top  of  the  Robot  Operating  System  (ROS).  This 
framework  [3]  enabled  empirical  comparison  of  belief  propagation  coordination  against 
baseline  algorithms  (such  the  potential  field)  in  various  environments,  both  simulated  and  with 
real  robots. We  additionally  explored  using  this  same  MRF  model  for  coordination  problems 
over  other  forms  of  complex  networks,  where  global  solutions  result  from  local  coordination. 
Specifically  in  the  area  of  manifold  learning,  we  used  a  MRF-based  model  to  develop  BP- 
Isomap,  a  belief  propagation  version  of  Isomap  nonlinearity  dimension  reduction.  BP-lsomap 
[4]  probabilistically  extended  manifold  learning  towards  greater  robustness  to  noise  in 
neighborhood  selection.  This  work  has  also  contributed  software  packages  for  reinforcement 
learning  using  ROS  with  an  eye  to  distributed  coordination  using  graph  Laplacians  [5]. 

Multi-robot  Markov  Random  Fields 


An  MRF  is  a  graphical  model  that  factors  a  system  into  a  finite  set  of  observed  and  hidden,  or 
latent,  variables  with  pairwise  interactions  between  them.  Variables  in  our  case  are  vector- 
valued  random  variables,  meaning  that  they  are  multidimensional  probability  distributions. 
Each  agent  /'maintains  one  observed  and  one  hidden  variable.  The  observed  variable  yi 
represents  an  agent’s  perception  of  the  physical  world.  These  perceptions  are  derived  from  a 
robot’s  own  sensing  information  and  pertain  only  to  information  about  physical  objects  in  the 
world.  Local  evidence  only  includes  information  about  other  agents  in  a  mechanical  sense;  it 
does  not  contain  any  information  about  their  beliefs  or  intentions.  The  state,  which  is 
represented  by  the  hidden  variable  x„  is  a  vector  consisting  of  variables  important  to  the 
agent’s  behavior.  The  state  vector  can  contain  variables  representing  intended  actions  for  the 
agent,  plans  for  other  agents,  knowledge  about  the  environment  (such  as  localization 
information  or  object  classification),  or  other  high  level  information. 

In  the  multi-robot  case,  the  state  of  an  agent  is  conditional  on  its  observed  variable  and  the 
states  of  other  agents  whose  states  it  can  perceive.  From  the  perspective  of  a  single  agent, 
the  other  nodes  are  actually  the  perceived  states  of  other  agents.  Because  we  are  concerned 
with  collaboration  between  cooperative  agents,  there  is  no  reason  for  agents  to  deceive  each 
other  and  any  information  about  state  can  be  explicitly  communicated.  Considering  state  to  be 
perceived  state  will  simplify  the  model  and  allow  us  to  model  a  collaborative  team  as  a  single 
graph  with  agents  mutually  affecting  each  other  (Figure  1).  Given  the  observed  and  hidden 
variables,  a  pairwise  MRF  factors  inference  of  a  collaborative  team  action  xinto  two  functions: 


pairwise  compatibility  ifjjj  (xh  xj  between  each  agent  pair  (i,j)  and  local  evidence  </>,  (x„  y,). 


States  of  other  agents 


Compatibility- 

The  influence  of  agent 
2’s  state  on  agent  1 


State- The  agent’s  intentions 
and  knowledge  of  the  environment 


Local  Evidence- 

The  influence  of  the  agent’s 

local  perception  on  its  state 


The  agent’s  perception 
of  the  physical  world 

Figure  1:  The  Multi-robot  Markov  Random  Field  model  for  representing  the  coordination  of 
multiple  agents,  with  latent  beliefs  x,  about  the  team's  actions,  from  the  local  observations  of 
individual  robots  y,. 


Multi-robot  ROS  software  framework 


For  implementation  of  the  Multi-Robot  Markov  Random  Field,  we  have  produced:  1)  a  plug- 
and-play  framework  for  multi-robot  coordination  algorithms  with  existing  robot  middleware  and 
mesh  network  packages  and  2)  a  low-cost  implementation  of  the  proposed  framework  using 
commercial  off-the-shelf  hardware  and  open-source  software.  Our  framework  supports  multi¬ 
robot  coordination  through  a  mesh  network  which  allows  robots  to  communicate  information 
with  local  neighbors  to  produce  desired  global  behavior.  We  have  used  an  implementation  of 
the  proposed  framework  to  conduct  experiments  in  static  unmapped  physical  environments 
both  indoors  and  outdoors.  Our  implementation  of  the  proposed  framework  is  available  to  the 
community  as  an  experimental  resource.  Our  aim  is  that  by  providing  these  systems  other 
groups  can  leverage  our  work  to  test  their  own  complex  network  systems  and  algorithms  in 
the  physical  world.  This  framework  has  been  released  as  open-source  through  the  Brown 
University  ROS  repository,  brown-ros-pkg  (http://code.google.eom/p/brown-ros-pkg/). 

Initially  released  as  the  meshnet  ROS  package,  we  implemented  mesh  networking 
capabilities  for  multi-robot  systems,  enabling  autonomous  coordination  and  remote  user 
control  over  mobile  ad  hoc  networks.  This  work  combined  the  ROS  robot  middleware 
framework  (http://ros.org)  and  B.A.T.M.A.N.,  an  ad  hoc  routing  protocol  (http://www.open- 
mesh.net/),  with  custom  data  channeling  code.  This  meshnet  package  was  later  modified  to 
use  OLSR  (http://www.olsr.org/).  which  proved  to  provide  a  better  interface  for  mesh 
networking  capability.  Illustrated  in  Figure  2,  an  initial  3-robot  test  for  teleoperation  and  video 
forwarding  through  the  robot  mesh  network  can  be  viewed  online  at 
http://www.flickr.com/photos/brownrobotics/46049301 1 2/. 


Figure  2:  Snapshot  sequences  from  3-robot  mobile  mesh  network  test.  (Left)  the  teleoperator 
at  the  base  station  controls  a  lead  robot  to  explore  an  office  environment  (right)  using  video 
from  its  camera.  (Middle)  intermediate  robot  nodes  connect  the  lead  robot  to  the  base  station 
when  they  are  outside  direct  communications  range. 


Building  on  this  meshnet,  we  were  able  to  implement  standard  potential  field  coordination  for 
decentralized  robot  coverage  scenarios.  Highlighted  in  Figure  3,  the  system  was  able  to 
perform  wireless  coverage  for  situations  with  up  to  8  commodity  off-the-shelf  robots  over 
larger  areas  than  typical  robot  coordination  research  using  only  wireless  signal  strength  for 
sensing.  Using  the  publish-subscribe  messaging  of  ROS,  a  multi-robot  coordination  algorithm 
can  interface  with  and  run  on  multi-robot  platforms  by  subscribing  to  appropriate  input  topics 
(e.g.,  robot  pose,  wifi  signal  strength)  and  publishing  appropriate  output  topics  (e.g.,  robot 
movement  commands).  Support  packages  in  ROS  and  OLSR  handle  interfacing  with  robot 
hardware  and  mesh  network,  respectively,  providing  an  abstraction  layer  for  coordination 
algorithms. 


Figure  3:  Robots  in  our  team  consist  of  off-the-shelf  iRobot  Create  with  a  commodity  netbook 
(left).  Using  only  signal  strength  for  measurement,  coordination  occurs  over  ac  hoc  networks 
to  cover  larger  spaces,  such  as  the  3rd  Floor  of  the  Brown  CIT  building  (middle,  right). 


Released  as  part  of  the  Coordinating  Robot  Networks  using  Belief  Propagation  (crnbp)  ROS 
package,  we  extended  our  meshnet  robot  coordination  framework  to  faithfully  represent  and 
provide  comparisons  for  belief  propagation  with  Multi-robot  Markov  Random  Fields.  This 
package  provides  autonomous  localization  for  each  robot  (using  particle  filtering  on  a  known 
map)  and  belief  propagation  modules  for  autonomous  coordination.  Localization  modules 
provide  belief  distributions  about  the  location  of  a  robot  in  the  environment  based  on  its  local 
sensing  and  wireless  signal  strength  to  other  robots.  These  distributions  over  location  inform 
the  inference  over  coordination,  resulting  in  distributions  for  the  desired  location  of  each  robot 
for  coverage.  Estimates  are  taken  from  target  location  distributions  to  generate  and  update 
motor  plans  for  each  robot.  Shown  in  Figure  4,  we  have  tested  the  crnbp  package  with  robot 
teams  of  up  to  22  robots,  including  the  iRobot  Create-based  mobility  platform  (shown  in 


Figure  2)  as  well  as  the  AR. Drone  quadrotor  helicopter  (released  as  through  our 
ardrone_brown  and  ardrone_nav  ROS  packages). 


and  22  (bottom  row)  robots  on  the  first  floor  of  the  Brown  CIT  building.  Each  row  shows  the 
starting  and  converged  team  configurations.  Each  picture  displays  the  locations  of  each 
robot  with  edges  showing  robots  that  communicating. 


Summary 


For  this  project,  we  have  produced  the  Multi-robot  Markov  Random  Field  as  a  unifying  model 
for  multi-robot  coordination  algorithms.  This  model  allows  for  the  creation  and  comparison  of 
multi-robot  coordination  algorithms  in  a  generative  framework  for  expression  of  (but  not 
limited  to)  probabilistic  inference.  We  have  proposed  coordination  algorithms  using 
probabilistic  belief  propagation  to  improve  robustness  in  the  face  of  perceptual  uncertainty. 
Our  algorithms  have  been  implemented  and  released  open-source  through  the  ROS  robot 
middleware  framework. 
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Software  Releases 


The  following  software  packages  resulting  from  this  work  have  been  released  open-source  for  use  with 
ROS,  the  Robot  Operating  System: 

•  ardrone_brown  :  driver  for  Parrot  AR. Drone  quadrotor  helicopter 

o  (http://brown-ros-pkg.googlecode.com/svn/trunk/experimental/ardrone_brown/) 

•  ardrone_nav:  autonomous  quadrotor  navigation 

o  (http://brown-ros-pkg.googlecode.com/svn/trunk/experimental/ardrone_nav/) 

•  crnbp:  Coordinating  Robot  Networks  with  Belief  Propagation  ROS  framework 
o  (http://brown-ros-pkg.googlecode.com/svn/trunk/experimental/crnbp/) 

•  rl_glue :  ROS  framework  for  RL-Glue  reinforcement  learning 

o  (http://brown-ros-pkg.googlecode.com/svn/trunk/experimental/rlrobot/) 

•  meshnet:  multi-robot  coordination  framework  using  BATMAN  and  OLSR 
°  depreciated  due  to  crnbp 

(http://brown-ros-pkg.googlecode.com/svn/trunk/experimental/meshnet/) 
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